ENUMERATION OF TILINGS OF DIAMONDS AND HEXAGONS 

WITH DEFECTS 
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Abstract. We show how to count tilings of Aztec diamonds and hexagons with defects using 
determinants. In several cases these determinants can be evaluated in closed form. In particular, 
we obtain solutions to open problems 1, 2, and 10 in James Propp's list of problems on enumeration 
of matchings J2l|] . 



1. Introduction 

While studying dimer models, P. W. Kasteleyn [13] noticed that tilings of very simple figures 
by very simple tiles can be not only plausible physical models, but also starting points for some 
very interesting enumeration problems. Kasteleyn himself solved the problem of counting tilings 
of a rectangle by dominoes. He also found a general method (now known as Kasteleyn matrices) 
for computing the number of tilings of any bipartite planar graph in polynomial time. Kasteleyn's 
method has proven very useful for computational-experimental work, but it does not, of itself, 
provide proofs of closed formulas for specific enumeration problems. We shall see a few examples 
of problems for which Kasteleyn matrices alone are inadequate. 

By an (a, b, c, d, e, /) hexagon we mean a hexagon with sides of lengths a, b, c, d, e, /, and angles 
of 120 degrees, subdivided into equilateral triangles of unit side by lines parallel to the sides. We 
draw such a hexagon with the sides of lengths a, b, c, d, e, / in clockwise order, so that the side of 
length b is at the top and the side of length e is at the bottom. We shall use the term (a, b, c) 
hexagon for an (a, b, c, a, b, c) hexagon. Thus Figure || shows a (3, 4, 3) hexagon. 

An Aztec diamond of order n is the union of all unit squares with integral vertices contained 
within the region \x\ + \y\ < n + 1. Figure |l| shows an Aztec diamond of order 3. 

We are interested in tilings of hexagons with lozenges, which are rhombi with unit sides and angles 
of 120 and 60 degrees, and tilings of Aztec diamonds with dominoes, which are 1 by 2 rectangles. 
In particular, we shall examine three problems from James Propp's list of open problems on tilings 

[0- 

Problem 1 (Propp's Problem 1). Show that in the (2n — l,2n,2n — l) hexagon, the central vertical 
lozenge (consisting of the two innermost triangles) is covered by a lozenge in exactly one-third of 
the tilings. 

Problem 2 (Propp's Problem 2). Enumerate the lozenge-tilings of the region obtained from the 
(n, n + 1, n, n + l,n,n + 1) hexagon by removing the central triangle. 

Problem 3 (Propp's Problem 10). Find the number of domino tilings of a (2k— 1) by 2k undented 
Aztec rectangle with a square adjoining the central square removed, where the a by b undented Aztec 
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Figure 1. Aztec diamond of order 3 Figure 2. (3,4,3) hexagon 



rectangle is defined as the union of the squares bounded by x + y < 6+1, x + y > 6 — 2a— 1, 
y — x <b + l, y — x> —(b + 1). 

We have solved these three problems, not by using Kasteleyn matrices, but by choosing a new 
approach, which, while much less general than Kasteleyn matrices, is better suited for problems 
like these three. We can summarize our approach as follows: 

1. Find the number of tilings of half of a hexagon or half of a diamond, with dents at given 
places. This is not new: see |6| and 0. 

2. Express the number of tilings of the figure as a whole as a sum of squares of the expressions 
obtained in the first step. The sum's range depends on the "defects" (missing triangles or 
squares, fixed lozenges or dominos) given in the problem. 

3. Express the sum of squares as a Hankel determinant. 

4. Evaluate the Hankel determinant using continued fractions or Jacobi's theorem. 

C. Krattenthaler has been working on these problems at the same time as us, together with M. 
Ciucu || and S. Okada 19 . The solution to Problem 1 in || is literally orthogonal to ours: Ciucu 



and Krattenthaler slice the hexagon vertically rather than horizontally. More generally, Fulmek 
and Krattenthaler Q have counted tilings of an (n, to, n, n, to, n) hexagon that contain an arbitrary 
fixed rhombus on the symmetry axis that cuts through the sides of length m. Krattenthaler and 
Okada's solution jl9| to Problem 2 and Krattenthaler's solution jl6| to Problem 10 are much like 
ours in steps 1 and 2. Thereafter, they are based on identities for Schur functions, not Hankel 
determinants. The work of Krattenthaler and his coauthors and our work thus complement each 
other. 

2. From Tilings to Determinants 

First we note that a necessary and sufficient condition for an (a, b, c, d, e, /) hexagon to exist is 
that the parameters be nonnegative integers satisfying a — d — c^f — e — b. The number of upward 
pointing triangles minus the number of downward pointing triangles in an (a, b, c, d, e, /) hexagon 
is a — d. Then since every lozenge covers one upward pointing triangle and one downward pointing 
triangle, an (a, b, c, d, e, /) hexagon can be tiled by lozenges only if a = d, and this implies that that 
the hexagon is an (a, b, c) hexagon. Moreover, if we remove a — d upward pointing triangles from an 
(a, b, c, d, e, /) hexagon with a > d, then the remaining figure will have as many upward pointing 
as downward pointing triangles. 

Definition 1. A (fc, o, k) upper semi-hexagon is the upper half of a (A:, q, k) hexagon having sides 
k,q,k,q + fc, i.e., a symmetric trapezium. A (k,q,k) lower semi-hexagon is defined similarly. A 
(fc, g, k) dented upper semi-hexagon is a (fc, q, k) semi-hexagon with k upward pointing triangles 
removed from the side of length q + k. (Figure shows a (3, 4, 3) dented upper semi-hexagon with 
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Figure 3. Dented 3 by 2 Aztec rectangle Figure 4. Dented (3,4,3) semi-hexagon 

dents at positions 1, 4, and 6). It will be convenient to use the term semi-hexagon for an upper 
semi-hexagon. 

Note that a (k, q, k) semi-hexagon is the same as a (k, q, k, 0, q + k, 0) hexagon, so removing k 
upward pointing triangles leaves a region with as many upward as downward triangles. 

Definition 2. An a by b dented Aztec rectangle is the union of the squares bounded by x+y < 6+1, 
x + y >b — 2a— 1, y — x <b, y — x > — (6+1), with the squares in positions tq < n < ■ ■ ■ < rt-i 
removed from the side given by y — x < b (see Figure ^). 

Before proceeding with our results on tilings, we first note some facts about the power sums 
P + 2-7 + • • • + m? that we will need later on. We omit the straightforward proofs. 
For any integer m and any nonnegative integer j we define by 

V H hm J , if m > 0; 

Si, { 0, if m = 0; 

_ {-iy +1 (cp + v h h (-m - iy), if m < o, 

where we interpret 0° as 1. 

Lemma 1. The numbers have the following properties: 

(1) For any integers p and q, with p < q, 

p> + (p + iy + ■■■ + <? = si-si_ v 

(2) Si = for all j and = for j > 0. 

(3) For to > 0, S ] _ m = (-lp'+ 1 5 J J „„ 1 . 

(4) For m > 0, S° m is given by the exponential generating function 

^ X J _ e x (e mx - 1) 

3=0 J 

(5) S J m is a polynomial in to of degree j + 1, with leading coefficient + 1). 

Next we prove two known results. First, we have a closed expression for the number of tilings of 
semi-hexagons with given dents, first stated in this form in J6[. This is equivalent to a well-known 
result on the enumeration of Gelfand patterns, as noted in [pLor on column-strict plane partitions. 
(See Knuth [l5| exercise 23, p. 71; solution, p. 593] for a proof similar to ours.) 

Lemma 2. The number of tilings of a (k, q, k) semi-hexagon with dents at positions < ro < • • • < 
rfe_i < q + k is 

0<Kj<k 
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where V n = 1! 2! 



IIo 



I0<i <j<r 

Proof. We proceed by induction on k. For the case k = 1, there is only one tiling, no matter where 
the solitary dent is. Hence the lemma holds for k = 1. 

Let us now assume the lemma holds for k. Suppose we have a tiling of a (k + 1, q, k + 1) with 
dents at < ro < • ■ ■ < r^ < q + k + 1. If we remove the bottom layer of lozenges from the dented 
side, we obtain a tiling of a (k,q,k) semi-hexagon with dents at < to < ■ ■ ■ < £fc— l < q + k, 
ri < tt < fj+i. For every such tiling of a (fc, q, k) semi-hexagon with dents at those places, there is 
exactly one tiling of the dented (fc, g, k) semi-hexagon. Hence 



T, 



E 



T, 



k,q,t 



E 



14- 



n & -*.•)• 



ri<ti<r i+1 Q<i<j<k 
I -\k-l 

, y 

V k -x I 1 lo 



+ i 

E 



14-i 
l 

"^7 



S: 



C3 

°r -l 



k-1 


fc-1 



where S 3 m = P + 2 J + • ■ • + m- 7 . In the second line of our calculations we can see that, since 
Ylo<i<j<k(ti ~ **) depends only on the differences between the t;'s, Tk+i, q , r depends only on the 
differences between the r^'s, not on their actual values. (It is also easy to see this combinatorially.) 



is a 



Hence it is sufficient to prove the formula in the case ro = 0. By Lemma |l|, S; 
polynomial in m of degree j + 1 with leading coefficient + 1) that vanishes at 0. Thus we can 



•j 

m—l 



reduce the determinant 
Hence 



k-i 



to 



fc-i 



by elementary column operations. 



i 



k+l,q,r 



r '+l 



l 

nr 2 • ■ 



i + i 

fe-i 



A— 1 



II 



J+l 



i+1 




Tk 



I fc-1 



14 

rrrg • • - He 
14 

w n 



'i+l 



n 

l<i<i<fe+l 



n) 



( r i -n), 

0<i<j<k+l 

since we assumed that ro = 0. Then by our observation the formula holds for all values of tq. 



□ 



3. Tilings of Dented Aztec Rectangles 

Definition 3. An a by b dented Aztec rectangle is the union of the squares bounded by x+y < 6+1, 
x + y > 6 — 2a — 1, y — x < b, y — x > —(6+1), with the squares in positions ro < r\ < ■ ■ ■ < r^-i 
removed from the side given by y — x < b (see Figure ||) . An a by 6 undented Aztec rectangle is an 
a by 6 + 1 dented Aztec rectangle with all squares on the side given by y — x < b removed. 
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Our next result counts dented Aztec rectangles. Another proof can be found in j?J. Just as 
tilings of dented hexagons correspond to Gelfand patterns, in [ffl] it is shown that tilings of dented 
Aztec rectangles correspond to monotone triangles, and in this context, a proof of the formula can 
be found in Q. 

Lemma 3. The number of tilings of an a by b dented Aztec rectangle with dents at < ro < • • • < 
r&_i < a is 

A a ,b,r = -y 11 fa -n), 

where V n — 1! 2! ■ • • n\. 

Proof. We proceed by induction on b. First we note that if n — r-i+i for some i, then the lemma 
asserts that A a ^, r — 0, which is correct. Although of no interest in itself, this case will be necessary 
for the induction. 

If b = 1, there is only one tiling, no matter where the one dent is. (In general, the number of 
dents has to be equal to b for the dented Aztec rectangle to be tileable.) Hence the lemma holds 
for 6 = 1. 

Let us now assume the lemma holds for b. Suppose we have a tiling of an a by 6 + 1 Aztec 
rectangle with dents at < ro < • • • < rj, < a. If we remove all dominoes with one or two squares 
on the dented long diagonal and the adjacent short diagonal, we obtain a tiling of an a by b Aztec 
rectangle with dents at < to < • • • < tf,—i < a, where r& < i& < r^+i- For every such tiling of 
an a by b Aztec rectangle with dents at those places, there are 2 m tilings of the a by b + 1 dented 
Aztec rectangle, where m is the cardinality of {k : Tk < tk < rk+i}. 

Next we show that this implies 

le{0,l} b r k <tk-lh<Tk+i 

This follows from the fact that if < tk < rk+i then r^ < tk — lk< r^ + i if Ik is cither or 1, but if 
Tk = tk then this inequality holds only for Ik = and if rk = tfc+i, it holds only for Ik = 1. Thus the 
number of different possible values of I corresponding to a given sequence r, is 2 m , where m is the 
cardinality of {k : rk < tk < f"fe+l}- Moreover, if for some I G {0, l} 6 , t satisfies rk < tk — h < i"k+l 
for all i, then we must have to < t\ < • •• < f(,-i, so all terms A a ^,t that occur in |l| either have 
to < ■ ■ ■ < tb-i or are zero; in either case they are covered by the induction hypothesis. 
Hence 

A a ,b+l,r = A at b,t 
le{0.1} b r k <t k -l k <r k + 1 
,6(6-1) 

e e n ( 2 ) 

iS{0,l} b r k +l k <t k <r k + 1 +l k 0<i<]<b 
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Vb- 


1 



E E 

;6{0,l} b r k +l k <t k <r k + 1 +l k 



6-1 




E l^n+i+ii-i _ 5 'r ! +; ! -i 
ie{o,i} b 
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where S 3 m = P +2 J + • • • + m J . Now if u(i,j, k) is any function denned for < i,j < b, < k < 1, 
then since a determinant is a linear function of its rows, we have 



E K*'J'^)lo = \u{i,j > 0) + u(i,j,l)\ 
ie{o,i} b 



6-1 



Thus 



.4 



a,b+l,r 



b(b-l) 

V b -i 



6-1 



(^r i+ i-l + ^n+i) ~~ (^r -l + ^ro ) 



6-1 



By (||), we can see that, since Ylo<i<j<b(tj depends only on the differences between the ifc's, 
A a ,b+i,r depends only on the differences between the r/c's, not on their actual values. Hence we 
may assume that ro = 0. Since Sj n _ 1 + S} n — (SLi + Sq) = Sf n _ 1 + S ] m — S 3 _ 1 is a polynomial in m 
of degree j + 1 with leading coefficient 2/(j + 1) that vanishes at 0, we can reduce the determinant 



6-1 



to 



2^7C7 + 1) 



6-1 



by elementary column operations. Hence 



A 



a,6+l,r 



2^ 
V b -i 



2 2 



2r 



j'+i 

i+l 



i + i 

r i+i 

'i+l 



6-1 





6-1 



(b+iyb 

1 2 r\T2---Tb 

- (b+l)b 

2 2 rir 2 • • • n 



'i+l 



1 6-1 

lo 



15, 



n ( r i-^) 

l<i<j<6+l 



(6+l)b 
2 2 



n (' 



0<i<j<b+l 



□ 



4. From hexagons to Determinants 



We now compute the number of tilings of a (k, q, k) hexagon with restrictions on where vertical 
lozenges may cross the horizontal symmetry axis. 

Proposition 4. Let L be a subset of {0, 1, . . . , k + q — 1}. Then the number of tilings of a (fc, q, k) 
hexagon in which the set of indices of the vertical lozenges crossing the q + k-long symmetry axis is 
a subset of L is 



I 



i+j 



fc-i 



Proof. We first recall that by the Binet-Cauchy theorem jUj, p. 9], if M is any k by n matrix and 
M* is its transpose, then the determinant of MM* is equal to the sum of the squares of the k by k 
minors of M. 
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The number of tilings of a (k, q, k) hexagon in which the indices of the vertical lozenges crossing 
the q + /c-long symmetry axis are ro < t\ < ■ ■ ■ < r^-i is clearly 



rpl 

k,q,r 



1 



ik-l\ 2 



V 2 



no 



Thus the number of tilings to be counted is the sum of T£ r over all ro < r\ < 



each rj is in L. Now suppose that the elements of L are lo < h < 
by n matrix {K)o<i<k, o<j<n- Then by the Binet-Cauchy theorem, 



< rfc_i where 
• < ln-x an d let M be the k 



Vr 2 

/ j k,q,r 



I MM* I = 



V, 



fc-i 



leL 



fc-i 



□ 



Note that since the numbers Tfc ig r depend only on the differences of the rj, the determinant 
in Proposition [| depends only on the differences of the elements of L; thus we may shift all the 
elements of L by the same amount without changing the determinant. This observation will be 
useful later on: 

Lemma 5. For any finite set L of numbers and any number u, 



□ 





fc-i 




fc-i 










leL 











Proposition 6. The number of tilings of a (fc, 2n + 1 — k, k, k + 1, 2n — k, k + 1) hexagon with a 
triangle removed below the center of the horizontal line dividing the two "hemispheres" is 

fc-i 

n+j+i 1 



V k -iV k 



^=1 



Proof. If we cut such a tiled hexagon into two parts by a horizontal line through the middle vertices, 
and then remove the lozenges that are bisected by this line, we obtain a tiling of a (k, 2n + 1 — k, k) 
upper semi-hexagon with dents at points ro < r± < ■ ■ ■ < r^-i, and a tiling of a (k+ 1, 2n— k, k+ 1) 
lower semi-hexagon with dents at points ro < r\ < ■ ■ ■ < r k -\ and at the center. Since the formula 
in Lemma || depends only on the differences among the r, , we can make zero lie on the center of 
horizontal line dividing the two "hemispheres" of the hexagon. Thus, we have — n < r < r% < 
■ ■ < ffc-i <n,n^ 0. 

The number of tilings of the upper semi-hexagon is 

vT7 n to-*), 

0<i<j<fc 

and the number of tilings of the lower semi-hexagon is 

k n n n- 

0<i<j<k 0<i<k 

Hence the number of tilings of the hexagon for given —n < ro < 

1 ,\ t l fc-1 so TT , , 1 



Vk-iVk 

(Note that this vanishes whenever rj 



( i;; t n " 

0<i<fc 



for some j.) 



( b 



1/2, 



< i"k— i < rt is 
fc-i 
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Now let M be the k by 2n + 1 matrix f)o<i<k,-n<j<n- Then by the Binet-Cauchy theorem, 
the number of tilings of the hexagon is 

2 



E 



i 



-n<ro<- ■ -<r n - ± <n 



V 3 \ ll2 r) 



fc-i 
o 



1 








i 




-iV k 
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r, 





MM 



E 

-n<Kn 



fc-1 



(1 + 



fc-1 



□ 



5. From Aztec rectangles to determinants 

For our next result, we use the following lemma, which is analogous to the Binet-Cauchy theorem. 

Lemma 7. Let U = (uij) be a 2k by k matrix, with rows indexed from to 2k — 1 and columns from 
to k — 1. For each k-subset A of {0, 1, . . . ,2k — 1}, Zei U a be the k by k minor of U corresponding 
to the rows in A and all columns, and let A be the complement of A in {0, 1, . . . , 2k — 1}. Then 

U A U A = 2 k \u 2id \u 2i+ltj l^ 1 . 

AC{0,... ,2fc-l} 

\A\ = k 

Proof. The lemma is a direct consequence of a result of Propp and Stanley Theorem 2] . More 
precisely, the lemma follows from their result when we sum over all possibilities for A*. (As noted 
by Propp and Stanley, their result is a special case of a theorem of Sylvester p3.) □ 



Proposition 8. The number of tilings of an a by b undented Aztec rectangle, where a < b < 2a + l, 
and b = 2k + I, with squares with indices ro < r\ < ■ ■ ■ < rb- a -l missing from a diagonal of length 
a + 1 going through the central square, is 




0<j<2k+l-a 



where to < t\ < ■ ■ ■ < t2 a -2h-i ore the elements of {0, 1, ■ • • a} — {rg, n, . . . , r2k-a}- 

Proof. Every tiling of the undented Aztec rectangle with missing squares can be subdivided into 
a tiling of two a by k + 1 dented Aztec rectangles with sets of dents disjoint from each other and 
from {r ,ri, . . . ,r 2 k-a}- Let T = {0, 1, • • • a} - {r ,ri, . . . ,r 2 k-a}, and let t < h < ■ ■ ■ < t 2a - 2k -i 
be the elements of T. Then the number of tilings of the undented Aztec rectangle with missing 
squares is 

2fc(fc+i) 

y 2 n ( r *- r j) n \^- r j\ 

k 0<j<i<2k+l-a 0<i<2a-2k 

0<j<2fc+l-a 

x e n (*i-*o) n (*i-*o)» 

\P\—a-k t <ti to<ti 
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which, written with determinants instead of products, is 
2*(*+i) 



v k 2 



n (n- r i) n e N 



0<j<i<2fc+l-a 



0<i<2a-2fc 
0<j'<2fc+l-a 



PCT 
|P|=o-fc 



t-fc-1 



l-fc-1 



where po < pi < ■ ■ ■ < p a -k-i are the elements of P and qo < q\ < ■ ■ ■ < q a -k-i ar e the elements 
of T — P. Applying Lemma yields the theorem. □ 

We can prove the following proposition in exactly the same way. 

Proposition 9. The number of tilings of an a by b undented Aztec rectangle, a < b < 2a + 1, 
b = 2k, with squares with indices ro < n < • • • < rt- a -i missing from a diagonal of length a + 1 
touching the central square is 



2& — fc+a 
V k -!V k 



n ( r *- r j) n i** 



0<i<2a-2fc+l 
0<j<2k~a 







a — k 




a — k—1 


) 


t j 

L 2i 





b 2i+l 






x 0<i<i<2fc+l-a 

where to < t\ < ■ ■ ■ < t2 a -2k are the elements of {0, 1, • • • a} — {ro, r\, . . . , r2fe- a -i}- 

6. Computing Determinants of Aztec Rectangles: A Special Case 

We can now solve Problem || using Proposition || with a = 2k — 1, b = 2fc, r = — 1. The 
number of tilings is 



2& — fc+a 



n 

2a-2fc 
<2fc— . 



0<i<2a-2fc+l 
0<j<2fc-a 



t '2i+l 



2fe 2 +fe-i 

Vk-iVk 
2k 2 +k-i 

Vk-2Vk-l 
2k 2 +k-l 



Vk-2Vk-l 



0<i<2fc-l 



a — 


fc-1 













fc-1 




L 2i 





, '2i+l 



fc-2 




(fc-l)!fc! \t{ 



2 1 



fc-1 



b 2i+l 



ifc-2 



][][ (*2ji — *2j ) Jl (*2ji + l _ *2j + l) 



o<io<ii<fc 



o<io<ii<fe-i 



J| (%i -*2jo) II -%o+l)- 



o<io<ii<fe-i 



o<io<ii<fc 
For fc = 2(7, we have 

JJ (*2j! - t 2 j ) 
0<Jo<Ji<fc 

= [] ( 2 -h-2jo) I] ((2?+l + 2j 1 )-(2g+l + 2j )) 

o<j'o<j'i<g 0<io<ii<g 

x {J ((2 9 + l + 2 J i)-(2j )) 
0<jo ,ji<g 

= (2"- 1 4"- 2 ---(2g-2)) 2 

x 3 • 5 2 • • • (2g - l) q - 1 (2q + l)"(2g + 3)^ ■ ■ ■ (4g - 1) 
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JJ (<2ji+l - *2jo + l) 
0<Jo<Ji<fe-l 

II ((2ji + 1) - (2j + 1)) 1] ((2« + 2ji)-(2g + 2j )) 

0<jo<ji<9-l 0<jo<Ji<9 
X J] (( 2 g + 2ji) - (2j + 1)) 

0<io<9-l 
0<ji<g 

= (2 q - 2 A q - 3 ■ • ■ (2g - 4))(2 9 - 1 4«- 2 ■ • • (2« - 2)) 

x 3 • 5 2 • • • (2q - l) q - 1 (2q + I) 9 " 1 ■ • • (4q - 3). 

For k = 2q + 1, we have 

II (*2ji-*2j ) 
0<Jo<Ji<fe 

= J] ( 2 J'i- 2 J'o) II ((2« + l + 2ji)-(2g+l + 2jo)) 

0<jo<ji<9 0<j o <ji<q+l 

x I] (( 2 g + l + 2j 1 )-(2io)) 
o<j <q 

0<ji<g+l 

= (2«- 1 4' J - 2 ■ • • (2q - 2))(2 <? 4«- 1 • • • (2qj) 

x 3 • 5 2 • • • (2q - l) q - 1 (2q + l) q (2q + 3) q ■ ■ ■ (4g + 1) 



and 



(*2ji+l -*2j +l) 

0<jo<ji<fe-l 

= [] ((2.7i + 1) - (2.70 + 1)) 1] ((2g + 2 + 2j 1 )-(2 g + 2 + 2jo)) 

0<jo<jl<q 0<j o <ji<q 

x n ((2g + 2 + 2ji)-(2j + l)) 

o<j ,ji<q 
= (2 q -H q - 2 ■ ■ ■ (2q - 2)) 2 

x 3 • 5 2 • • • (2q - l) q - 1 (2q + l) q (2q + 3) 9 " 1 ■ ■ ■ (4g - 1) 
Therefore, for fc = 2q the number of tilings is 

2 (2 9 ) 2 +2g-l 

-2 4q -H 4q - 9 ■ ■ ■ (2q - 2f 



x 3 2 5 4 ■■■(2q- l) 2q - 2 (2q + l) 2q - 1 (2q + 3) 2q - 3 ■ ■ ■ (4q - 3) 3 (4<z - 1), 
and for k = 2q + 1 the number of tilings is 

2 (2 9 +l) 2 + (2 9 +l)-l 

2 4 9 -3 4 4 9 -7 . . . {2q _ 2)B(2g) 

x 3 2 5 4 • • • (2(7 - l) 2q - 2 {2q + l) 2 «(2 g + 3) 29 - 1 ■ ■ ■ (4g - l) 3 (4g + 1). 
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7. Computing Determinants: Hexagons 



In this section we solve Propp's Problem 1, and more generally, we count tilings of a (2m — 
1, 2n, 2m — 1) or (2m, 2n— 1, 2m) hexagon with a vertical lozenge at the center. (A (k, q, k) hexagon 
has a central vertical lozenge if and only if k + q is odd.) 

Lemma 10. The number of tilings of a (2m — 1, 2n, 2m — 1) hexagon with a vertical lozenge in the 
center is 



1 



V 2 

v 2m-2 



(i + (-iy+^V ! 



The number of tilings of a (2m, 2n — 1, 2m) hexagon with a vertical lozenge in the center is 



1 



V 2 

'9.™ 



(l + (-l)^)S;+fn-l 



2m-l 



Proof. By Proposition ^, the number of tilings of a (2m — 1, 2n, 2m — 1) hexagon is 



V 2 

v 2m-2 



2m+2n— 2 



E 

2=0 



By Lemma ^, this determinant is equal to 



m+n — 1 



E V+1 

l — — m — n-\-l 



(i + (-i) 4+J )^„-i + ^ +J 



2m— 2 



(1 + ( l) i+J ')S' m + n _ 1 



2m— 2 



(l + (-l) W )S£L_! 



2m-2 



where is 1 if fe = and is otherwise. 

It also follows from Proposition |^ that the number of tilings of a (2m — 1, 2n, 2m — 1) hexagon 
that do not have a vertical lozenge in the center is 

2m-2 

0<Z<2m+2n-2 
l^m-\-n— 1 



i 



v 2 

v 2m-2 



By Lemma |5] this determinant is equal to 



E 



-m — n+l<Z<m+n— 1 
1^0 



(i + (-i) i+i )^„-i 



2m-2 




We find the number of tilings that do have a vertical lozenge in the center by subtracting from the 
total number of tilings the number of tilings that do not have a lozenge in the center. 

The formula for (2m, 2n — 1, 2m) hexagons is derived similarly. □ 

As a first step in evaluating the determinants in Lemma n(J, we evaluate the determinant 



\si +j 



k-l 



P 10 



It is interesting to note that this determinant was evaluated by Zavrotsky l27[ in the 



course of his research on minimum square sums, and we follow his proof. 
Lemma 11. 

v k-l 



I ci+3 I k 1 
\°P lo 



V2fc-1 

_ Vp+k-iVp-k-iVk-i 
where = V + 2 l + • • • + p l and V p = l!2! • • -pi 



(p - k + 1) • • • (p - l) fe - + l) fe_1 • • • (P + A - 1) 
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Proof (Zavrotsky |27|). If p is a positive integer, we can express the matrix (S 1 *" 1 "^)^ -1 as the product 
of a k by n matrix and an p by k matrix, as in the proof of Proposition Since the rank of an p 
by k matrix is at most p, the rank of the matrix (S^^)^" 1 is at most p. Moreover, this holds also 
for p = 0. 

Now let (ai.j(A))Q _1 be a matrix whose entries are polynomials in A. It is known H p. 17] that if, 
for some value Ao of A, the matrix (aij (Ao))g _1 has rank at most m, where m < k, then |dj,j(A)L 1 
is divisible by (A — Ao) fc ~ m as a polynomial in A. 

By Lemma [l], there is a polynomial S\ in A whose value at A = p is S*. Since the rank of 

(^m^O 1 is at m0st m i 



k-l 



is divisible by (A — m) m for < m < k. 



Since S l _ m = (— l) t+1 S* n _ 1 when i > 0, it follows that the rank of (5'!^)o _ 1S a ^ m °st one 

k-l 

is divisible by (A + m) k ~ m for 



more than the rank of (<S^jJ!i)q 1 ; i.e., at most m. Thus 
1 < m < k. 



Since S l x is a polynomial in A of degree i + 1, 

k-l 

is equal to a constant times 



k-l 



is a polynomial in A of degree k 2 . Hence 



(A - k + 1) • • ■ (A - l) fc_1 A (A + l) fc - 1 (A + 2) 



\fc-2 



(A + fc-1). 



fc-i 



Since S l x has leading coefficient + 1), and, by ]3|, p. 425], the determinant + j + 1)| 
(a Hilbert determinant) is equal to V^_ 1 /V2k-i, we may compare leading coefficients and that the 
constant is V^^/V-ik-i- D 



Corollary 12. The number of tilings of a (k,q,k) hexagon is 

V 2 k+q-lVq-lV k 2 - 1 

In particular, the number of tilings of a (2m — 1, 2n, 2m — 1) hexagon is 

V4m+2n-3^2n-lV2m-2 
^2 2 m+2n-2^4m-3 

and the number of tilings of a (2m, 2n — 1, 2m) hexagon is 

+ 2 n - 2 V2 n - 2 Vjj m - 1 

Proof. By Proposition ^ and Lemma |5[ the number of tilings of a (fc, q, k) hexagon is 

1 



V 2 

v k-l 



qi+j 

°k+q 



The result then follows from Lemma [ll]. 



□ 



It is also possible, as shown in ||, to derive the formula for the number of tilings of an (a, 6, c) 
hexagon directly from Lemma |^. 

Next we prove a general theorem on Hankel determinants that allows us to evaluate the deter- 
minants in Lemma |l0|. 

Proposition 13. Let {diji^o ^ e a sequence, and let 

H s {k) 



ifc-i 

a (i+j+s)/2\ ) 
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for k>l, with H s (0) = 1, where we take aj to be if i is not an integer. Define Xk inductively by 

H a (k + l) = X h +1 X k 1 ---X kl 

so that Xq = Hq(1) = ao and 

H Q (k - l)H Q (k + 1) 



A k 



for k > 1. Then 

Proof. Define M r (k) by 
It is easy to see that 
Then 



Lfe/2J 3 



H 2 (k) = X^H (k + 1) £ ft ^± (3) 
^ t\ A ^ 



M r (fc) = |a f+ j +r | 1 



H 2r {k) = M r (\k/2])M r+1 ([k/2\). (4) 
H 2 {2m + 1) Mi(m+1) H (2to + 2) 



H 2 (2m) Mi(m) ff (2m+l)' 

Thus it suffices to prove (||) for fc = 2m. 

Since @ holds for k = 0, to prove it for even fc we need only show that for m > 1, 

ff 2 (2m) H 2 (2to - 2) _ x fl A 2i _i 

A o 11 



F (2m + 1) ff (2m-l) u A± a 2j 



i=l 

Using (Q), we may write the identity to be proved as 

M 2 (m) _ M 2 (m - 1) _ AiA 3 ■ ■ ■ A 2m -i 
M (m+1) M (m) ~ A A 2 ---A 2m 
To prove (^), we use Jacobi's identity (ll], pp. 594-595], 

(Mi (to)) 2 - M (to,)M 2 (to) + M (m + 1)M 2 (to - 1) = 0. 
Dividing both sides by Mo(m)Mo(m + 1), we may rewrite Jacobi's identity as 

M 2 (m) M 2 (m-1)_ Mi{m) 2 
M (m + 1) M (to) ~~ M (m)M (m + 1) ' 

To complete the proof we need to express the right side of (0) in terms of the Aj 
We have 

M (m) _ # (2m - 1) _ H (2to - 2) 

A 2m _ 2 - 



M (m - 1) i? (2TO - 2) H (2m - 3) 

ff (2m-3)_. Mo(m-l) 

— A 2m - 2 A 2m _3 — — A 2m _ 2 A 2m _3— — — . 

Ho{2m — 4) Mo{m — 2) 

Since M o (0) = 1 and M Q (1) — A , this gives 

Mo(m) 

— — — - A Ai • ■ • A 2m _ 2 , 

M (m - 1) 

and thus 

M (m) = A ra (A 1 A 2 ) m - 1 • • • (A 2m _ 3 A 2m _ 2 ) 



(5) 



(G) 
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Similarly, we can show that 

Mi(m) = (AoAl)'™ • ■ • (A2m-4A2m-3) 2 (A2m-2A2m-l)- 

Making these substitutions in the right side of (||) yields (ph, completing the proof. □ 

Note. There is a simple combinatorial proof of Proposition ^ in which the determinant is inter- 
preted as counting nonintersecting paths; see Viennot |25|, Chapter IV]. 

We now apply Proposition |l3| to evaluate the determinant |(1 + (— 1)* +J ')S£ +J '| . 
Proposition 14. The determinant |(1 + (— l)' l+: >)S p +J f L is equal to 

i V2 P+ k + iV2 P - k -iV h 4 L ^ J (-p)j (p + i)j 

2 P +1 Vi p V 2k+1 ^ (i)2 ( i )j( 3 +p)j( i_ p)/ 

where (a)j = a(a + 1) • • • (a + j — 1). 
Proof. Let us set a 4 = + 5;. Then 



1 1 0, if i is odd v v ; ; P 



With the notation of Proposition [L3| the determinant to be evaluated is 

\ a (i+])/2\ 1 - | a (i+i+2)/2 | 



= H 2 (k). 

Thus by Proposition |13| we can express the value of this determinant in terms of the values of the 
corresponding determinants Ho(k). 
We have 



(i + (-ins* + s t = £ i\ 



i=-t 

ifc-i 



qi+3 
°2p+l 



k-l 

. This determinant 

o 



so by Lemma [| the determinant Ho(k) — |o(i+j)/2|o 1 ^ s ecma l to 
may be evaluated by Lemma [y], which gives 

. . V 2v+ kV 2p -kV£_ 1 
^o(fc) -j- . 

V 2p V 2 k-l 

Therefore, with A& as in Proposition |l3|, we have Ao = a = 2S p = 2p+ 1, and for k > 0, 

_ fc 2 (2p + fc+l)(2p-fc+l) 
fe 4 (2k- l)(2k+ 1) 

Thus by Proposition [l3[ we have 

Lfe/2J j . 

H 2 (fc) = A -^o(fe + l) e n^p 

3=0 i=i 21 



1 Vap+fc+i^p-fc-iV^ 4 L v (Pi (Pi (-P)j 



where (a)j = a(a + 1) ■ ■ • (a + j — 1). □ 

We can now combine Lemma [lO] with the determinant evaluation of Proposition |l4| to count 
tilings of hexagons with a vertical lozenge in the center: 
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Theorem 15. The number of tilings of a (2m — 1, 2n, 2m — 1) hexagon with a vertical lozenge in 
the center is 

^4m+2n-3V2»-lV r 2m-2 (|)| (f )j i 1 ~ W ~ ")j ( TO + ")j 

(2m + 2n - l)V 2 % l+2n _ 2 V 4m - 3 (l) 2 {\) 3 (I + m + n), (| - m - n), ' 
and i/ie number of tilings of a (2m, 2n — 1, 2m) hexagon with a vertical lozenge in the center is 

^4m+2»-2V2»-2V r 2m-l C 1 ~ TO ~ n )j ( TO + ")j 

(2m + 2n - l)V 2 2 m+2 „_ 2 ^ 4m -i ^ (l) 2 (§ + m + n)j (| - m - n) j ' 

To finish the solution of Propp's Problem 1, we need only evaluate the sum in Theorem [l5| in 
the case m = n. To do this we use the Wilf-Zeilberger (WZ) method p6[. 



Lemma 16. 



Proof. Let 



- 1 (I) 2 (|) t (l-2n), (2n)j _ 4n - 1 
^ (1) 2 (±)^ + 2n)«(f-2n) i 3 ' 

1 (|)?(l)i(l-2n) 4 (271), 



Q(n, i) 



4n-l(l)?(l)i(5 + 2n) 4 (f-2n) 4 



We want to prove that 



Since this identity is clearly true for n = 1, it is sufficient to prove that 

n n — 1 

^Q(n+l,i)-5^Q(n,i) = 

j=0 i=0 

for n > 1. 

To apply the WZ method, we must first find a function U(n,i) such that 

J7(n, i + 1) - U(n, i) = Q(n + 1, i) - Q(n, (7) 
With the help of Maple, we find that if we set 

. . x z 2 (2i + l-4n)(l + 4n)(8n 2 + 4n-2i 2 +i + l) 

U [71 l) — OiTl 1) 

v ' ' (4i + l)(2i + l + 4n)(i-2n)(i-l-2n)(2n+l)rc v ' ' 

then (0) is satisfied. (Once we have this formula for U (n, i), the verification of (0) is straightforward.) 

Next, we sum identity (|t]) on i from to n — 1 and add Q(n + 1, n) to both sides. The left side 
telescopes, and we get 

n n—1 

Q(n + 1, n) + I7(n, n) - I7(n, 0) = ^ Q(n + 1, ») - Q(n, *)• (8) 

i=0 1=0 

But t/(ri, 0) = and we can easily check that Q(n + l,n) + U(n,n) = 0. Thus the left side of (||) 
is 0, hence so is the right side. □ 
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Note. The sum in Lemma 16 is a partial sum of a special case of Dougall's very- well-poised 5^4(1) 
sum j|, p. 25, eq. (3)]: if the upper limit of summation were 2n—l instead of n— 1, we would have a 
special case of Dougall's theorem. It is interesting to note that in Ciucu and Krattenthaler's solution 
of Propp's Problem 1, they used an analogous evaluation of a partial sum of the Pfaff-Saalschiitz 
theorem ||, eq. (2.1)]. 

We can now finish our solution to Propp's Problem 1: 

Theorem 17. In a (2n — 1, 2n, 2n — 1) or {2n, 2n — 1, 2n) hexagon, the two central triangles are 
covered by a lozenge in exactly one-third of the tilings. 

Proof. We compare the result of setting m = n in Corollary [jj] with the result of setting m = n in 
Theorem |l5] and evaluating the sum by Lemma [l6|. □ 



8. Computing More Determinants 

By Proposition ||, evaluating the Hankel determinant |(1 + (— 1)*)5^ +J+1 | 1 will solve Propp's 
Problem 2. To do this, we use the close connection between Hankel determinants and continued 
fractions that was implicit in our proof of Proposition [l^. The following lemma is equivalent to |l2[ 
Thm. 7.2]. 

Lemma 18. Let {ai}f^ be a sequence, and suppose that the generating function for the a\ has the 
continued fraction 

00 \ 

QiX = 



Xix 

1=0 1 



X 2 x 



Then 



X 3 x 



\ a (i+j)/z\o -^0^1 1 ---X k -1, 



where we take a r to be if r is not an integer. □ 

By Lemma if we can find the continued fraction for 

00 00 

£(i + (-im+v-/ 2 = E 2 ^ +V > 

then we can evaluate the corresponding Hankel determinant. 

The continued fraction in question is given by the following formula, which we prove in the next 
section. 



Proposition 19. 



E 25 « +v = — — 



[l 2 X 
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where 

fi = n(n + 1) 

H2i= -rr—z(n + i+l)(n-i), i>l 

4:1 + 2 

M2i+i = 4i - 2 [n + i + l)(n - z). 

Now from Proposition |6[ Lemma |l8|, and Proposition [l^, we obtain the solution to Propp's 
Problem 2: 

Theorem 20. The number of tilings of a (k, In + 1 — k, k, k+ 1, In — k, k + 1) hexagon without the 
central triangle is 

(n-q)(n-q+l) 5 ---n 4 i+ 1 (n+l) 4 i+ 1 ---(n + q + l) , , o , 

— /or fc = 2g + 1, 



2 (2g)(2 g +l)38(9-l)+2 5 8(< ? -2)+2 . . . (2q + 1)2 

(n - q + l) 3 • • • n^- x {n + l) 4 ^ 1 • • ■ (n + qf 

2(2g-l)(2g)38(g-l)-2 5 8(g-2)-2 . . . (2g _ 1)6 



/or = 2g. 



9. Proof of the continued fraction 



In this section we prove the continued fraction of Proposition [l9| . 

The exponential generating function for (1 + (— l)^ v )S 3 n is, by Lemma [IJ 



(e nx - l){e x - e~ nx ) 



3 



e x - 1 
^ sinh ^x sinh 
sinh | 



Now let L be the linear operator on formal power series defined by 

, OO OO 



t=0 7 1=0 



We note that L(f(x)) has the "formal" integral representation 



oc 



L(f(x)) = -I f{t)e-*/'dt, 



obtained by performing the integration term by term. If F(x) = L{f{x)), then this formula may 
be written as a Laplace transform 



F(l/z) = z f(t)e~ tz clt, 



and this is the form in which it is most often seen in the literature on continued fractions. 

The continued fraction we need is given by the following formula, in which n need not be an 
integer. The case in which n is a nonncgativc integer is clearly equivalent to Lemma UM. 
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Lemma 21. 

( sinh fx sinh ^x \ _ x 



L 



\ sinh | J fj,ix 



,•2 



where 

M2i = ^— + i + 1)0 - i) 

M2l+1 = 4z + 2 ^" + * + ^ n ~~ 

Lemma ^l] is one of several continued fractions for this function given by Lange [[l7|, pp. 259- 
260]. (A closely related continued fraction for the same function was given by Stieltjes |p3[ .) For 
completeness, we give here a self-contained proof: 

Lemma 22. Let fa,fi,fi,--- be formal power series in x with nonzero constant terms, and let 
ci, C2, • ■ ■ be constants such that for each k > 1, 

/fc ~ /fc-i = c k x 2 f k+1 . (9) 

XTien /or eac/i m > 1, 

/m 1 



m — 1 



l - 



2 



l 



Proof. Equation (^) is equivalent to 

fk 1 



Iterating this formula gives 



fk-1 fk+l 

1 - c k x z — — 

Jk 

fm 1 



fn 



2 



2 fm+n+1 



Taking the limit as n — > oo yields the lemma. □ 
Proof of Lemma 21. Let E 1 = L _1 , so that 



s OO \ OO j 



\j=0 ' j=0 



and suppose that with f k as in Lemma |22|, g k — E(x k f k ). Multiplying the recurrence (Q) by x 
and using the fact that if u(x) is divisible by x then 



fe-2 
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we find that (Q) is equivalent to 

dg k 



, 9k-x=Ck9h+x- (10) 
ax 



We now consider the case of Lemma E2l in which 



c 2i = ( n + * + ~ ^ 

c 2i+i = ^f^( n + 1 + _ *) 



We shall express a solution of recurrence (M) in terms of the hypergeometric series, defined by 



a, b 
c 



E 



(a) n {b)n n 
- z . 



" ! ( c )r 



where (u) n = u(u + 1) • • • (u + n — 1). 

We claim that a solution of recurrence ([LOO is 



(e* - 1)* 
fc! 



— nx tj 
e 2-^1 



k + l 



l-e x 



x 2*1 



LIJ + i. L!J 



fc + i 



1-e 3 



(11) 



The verification that gi defined by (|11|) really does satisfy (|10|) is a straightforward, but tedious, 
computation using the formula 



d F 

az 



a, b 
c 



, ab _ f a+ 1, b+ 1 
z|= T 2 M c + 1 



together with the contiguous relations for the hypergeometric series jj], p. 558]. (This computation 
was done with the help of Maple.) 

It is not hard to show that go — 1. We now evaluate 



9l = (e x - l)e' nx 2 F 1 

Using the easily verified fact that 

1,0 



1, 1 - n 
2 



1 - e x ) 2 F 1 



1, — n 
2 



l-e* 



2-^1 



z(0- 1) V(l -z) fi 



-- 1 ■ 



(12) 



we find that 



91 



'(e nx - l)(e( n+1 ) x - 1) 
n{n + l)(e x - 1) 



2 sinh fx sinh ' n ^x 



n(n + 1) 



sinh | 



Thus /o = 1, and 



/i = -L( 9l 



fn+U 



-L 



sinh fx sinh ^^p-x 
sinh | 



Substituting these values of /o and /i into the case to = 1 of Lemma |2^, and multiplying both sides 
by ("J 1 )^, completes the proof of Lemma ^l[ □ 
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It is clear from the recurrence ( |To| ) and the value of gi that gk is a rational function of e x and 
e nx . Although we won't need it, we can give an explicit formula that expresses gk in this form by 
applying to (pi) the formula 



iFi 



m + 1 , (3 
k + 1 



(-1)* 



A:! 



(-1)' 



.(1-*) 



k — rn — fi 



X 2*1 



— m, 1 — (3 
I — [3 — m + k 



(l-/3)fc- 
1 - z 



(1 - fc) m fc ^ 1 (/? - - fc + m )» ,,i 



(l-fe)i 



for fc > m, which can be proved by equating coefficients of powers of z on both sides. (Note that 
(p"2|) is the case m = 0, fc = 1.) 
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